Network-based pathology system with desktop slide scanner

ABSTRACT

A method for processing, saving and viewing a digital image of a microscope slide includes inserting a microscope slide into a digital slide scanner connected to an acquisition computer. A pre-scan formed from a plurality of image tiles uploaded to a network server while the pre-scan is being generated. The network server analyzes the image tiles in realtime to identify an area of interest. The acquisition computer generates a high magnification local scan of the area of interest. The local scan is formed from a plurality of local image tiles that are uploaded to the network server while the local scan is being generated. Each local image tile is viewable by a client computer in communication with the computer network while the plurality of local image tiles is being uploaded. A raw final image is then saved on the network server independent of the acquisition computer.

CROSS-REFERENCE

This application is a continuation of Ser. No. 14/234,013, filed May 19,2014, which is a U.S. National Phase of PCT/US2012/047527, filed Jul.20, 2012, which claims the benefit of U.S. Provisional Application No.61/509,946, filed Jul. 20, 2011, all of which are incorporated herein byreference in its entirety.

FIELD

The following description relates generally to slide scanners, and inparticular to desktop slide scanners for network-based pathology.

BACKGROUND

In order to diagnose a disease it is often necessary to examine tissuesamples at high magnification. By locating and identifying anomalousfeatures in a tissue sample, a pathologist can make a diagnosis, helpthe patient's physician select appropriate treatment and provideinformation on the efficacy of previous treatments. Pathologists aretherefore critical to the diagnosis and treatment of many diseases.

In general, pathologists often work at locations geographically distantfrom the hospital or clinic at which a tissue sample is taken. In thepast it was necessary to physically transport a tissue sample from thelocation of the patient to the pathologist, for example by express mailor courier. A pathologist would then prepare a slide specimen from thetissue sample and examine it under a microscope. However, physicallytransporting the tissue sample to the pathology laboratory may be timeconsuming, particularly if the patient is in a rural or remote area.Furthermore, if the tissue sample crosses a border, it must be inspectedby customs officials. Finally, in many areas such as third worldcountries there simply are not many pathologists, thereby making itnecessary for pathologists to spend an inordinate amount of timetravelling to different facilities. For patients who require immediatediagnosis, this is a serious drawback.

The advent of digital pathology helped to alleviate this problem. Indigital pathology, a high resolution digital scan of a slide is takenand this image is electronically transmitted to the pathologist foranalysis. A physician or technician can prepare slides from tissuesamples and create high resolution scans for off-site analysis by thepathologist. Furthermore, high volume slide scanners may scan dozens ofslides per scanning operation. Thus, dozens of different slide specimensfrom one or more nearby medical facilities may be sent to a singlelocation with a high volume slide scanner where digital images or“virtual slides” are created. These virtual slides are thenelectronically transmitted to appropriate pathologists over a computernetwork such as the internet.

Thus, digital pathology and high volume slide scanners have helpedstreamline pathological analysis by creating a hub to which all physicalslides in a region may be sent. The high volume scanner at the hub isthen used to electronically distribute virtual slides to pathologistsanywhere in the world almost instantly. In other words, it is no longernecessary to send individual slides to pathologists in a number ofspecialized fields. Instead, all slides may be sent to the location ofthe high volume slide scanner, which is typically relatively near themedical facility where the tissue sample was taken compared to thelocation of the appropriate pathologist

However, although digital pathology with high volume slide scanners isan improvement over older pathology methods, it is not withoutdrawbacks. First, existing high volume slide scanners are very large andexpensive, often costing several hundred thousand dollars. This cost maybe prohibitive, particularly in less wealthy countries and/or ruralareas. Additionally, high volume scanners are generally loaded withslides only once a day. If an anomaly is found in a particular slide, itcannot be immediately removed or rescanned at a higher resolution formore detailed analysis. Further, high volume scanners are typicallyallotted to physicians only as workload allows so a physician may haveto wait one or more days before it is possible to scan a new slide.

Finally, another drawback to conventional digital pathology is that veryhigh demands are placed on the computer acquiring the image from thescanner. The acquired images are typically several gigabytes in size,and thus a powerful computer is required to quickly process, manipulateand analyze the images. These computers are generally very expensive,making the combination of a high volume slide scanner and acquisitioncomputer cost prohibitive for many facilities.

In summary, high volume scanners are helpful for streamlining digitalpathology and handling a large number of slides at once, but thereremains a need for a smaller, more affordable, more flexible and moreresponsive digital pathology system.

SUMMARY OF THE INVENTION

The embodiments of a desktop slide scanner for cloud-based pathologydisclosed below satisfy these and other needs. The following simplifiedsummary is provided in order to provide a basic understanding of someaspects of the claimed subject matter. This summary is not an extensiveoverview, and is not intended to identify key/critical elements or todelineate the scope of the claimed subject matter. Its purpose is topresent some concepts in a simplified form as a prelude to the moredetailed description that is presented later.

In one aspect of the disclosed embodiments, a method for processing,saving and viewing a digital image of a microscope slide includesproviding an acquisition computer connected to a network server incommunication with a computer network. At least one microscope slide isinserted into a digital slide scanner connected to the acquisitioncomputer. The acquisition computer generates a pre-scan of themicroscope slide at a pre-scan magnification and a pre-scan resolution.The pre-scan is formed from a plurality of sequentially scanned imagetiles acquired by the acquisition computer. The acquisition computeruploads each of the plurality of image tiles to the network server aseach image tile is scanned. The network server analyzes the image tileswhile the pre-scan is being generated to identify an area of interest inthe pre-scan. The acquisition computer generates a local scan of thearea of interest at a second magnification higher than the pre-scanmagnification. The local scan is formed from a plurality of local imagetiles acquired by the acquisition computer. The acquisition computeruploads each of the plurality of local image tiles to the network serverwhile the local scan is being generated. Each local image tile isviewable by a client computer in communication with the computer networkwhile the plurality of local image tiles is being uploaded. The networkserver assembles a raw final image of the local scan from a mosaic ofthe plurality of local image tiles. The raw final image is then saved onthe network server independent of whether the raw final image is savedon the acquisition computer.

In some embodiments, the above method may also include sharpening eachlocal image tile to create a plurality of sharpened local image tileswhile the plurality of local image tiles is being uploaded. The networkserver assembles a sharpened final image from the plurality of sharpenedlocal image tiles while the plurality of local image tiles is beinguploaded. The client computer has the ability to select between the rawfinal image and the sharpened final image for immediate viewing withoutsaving the raw final image and/or the sharpened final image locally andwithout transferring the entire raw final image and/or the entiresharpened final image after the acquisition computer generates the localscan.

In other embodiments, a method for remotely analyzing a digital image ofa microscope slide includes providing an acquisition computer connectedto a network server in communication with a computer network. At leastone microscope slide is inserted into a digital slide scanner connectedto the acquisition computer. The digital slide scanner includes amicroscope and a microscope stage, and a pre-scan of a microscope slideon the microscope stage is generated at a first magnification. A clientcomputer remote from the acquisition computer and connected to thecomputer network is also provided. A pathologist remotely views thepre-scan on the client computer to identify areas of interest in themicroscope slide. The digital slide scanner generates a live stream ofthe areas of interest of the pre-scan at a second magnification greaterthan the first magnification. The pathologist then remotely instructsthe digital slide scanner to move the microscope stage to analyzedifferent portions of the areas of interest in real time.

The pathologist may use the client computer to remotely focus themicroscope on regions of varying depth in the areas of interest in realtime. Additionally, the desktop slide scanner may also include an inkerthat marks the microscope slide in the areas of interest identified bythe pathologist.

To the accomplishment of the foregoing and related ends, certainillustrative aspects are described herein in connection with thefollowing description and the annexed drawings. These aspects areindicative, however, of but a few of the various ways in which theprinciples of the claimed subject matter may be employed and the claimedsubject matter is intended to include all such aspects and theirequivalents. Other advantages and novel features may become apparentfrom the following detailed description when considered in conjunctionwith the drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic drawing showing the architecture of anetwork-based pathology system with desktop digital slide scanner.

FIG. 2 is a flow chart showing the process of uploading images of amicroscope slide to a network server for remote analysis.

FIG. 3 is a flow chart showing the process of live streaming images of amicroscope slide from a desktop digital slide scanner to a remoteclient.

DETAILED DESCRIPTION OF THE INVENTION

In one aspect of the disclosed embodiments, a method for processing,saving and viewing a digital image of a microscope slide includesinserting a microscope slide into a digital slide scanner connected toan acquisition computer. A pre-scan formed from a plurality of imagetiles uploaded to a network server while the pre-scan is beinggenerated. The network server analyzes the image tiles in real-time toidentify an area of interest. The acquisition computer generates a highmagnification local scan of the area of interest. The local scan isformed from a plurality of local image tiles that are uploaded to thenetwork server while the local scan is being generated. Each local imagetile is viewable by a client computer in communication with the computernetwork while the plurality of local image tiles is being uploaded. Araw final image is then saved on the network server independent ofwhether it is saved on the acquisition computer.

The presently disclosed embodiments facilitate fast and accuratepathological analysis of tissue samples taken from a patient. Aphysician and/or technician obtains a tissue sample and prepares amicroscope slide from the tissue sample in a conventional manner. Theslide is then brought to a facility with a digital slide scanner asdescribed below.

The digital slide scanner of the presently disclosed embodiments is muchsmaller than conventional high volume slide scanners, and ideally is adesktop slide scanner. Unlike high volume slide scanners, the desktopslide scanner only scans a handful of slides at a time. For example, thedesktop slide scanner may include a cartridge capable of accommodatingbetween one and ten slides depending on slide size, although cartridgescapable of holding more than ten slides are also contemplated. Once theprepared slides are placed in the cartridge, the cartridge is insertedinto the desktop slide scanner which can then be activated to prepare apre-scan of the slides.

The desktop slide scanner is connected to a local acquisition computerwhich is connected to a network server in communication with a computernetwork. Throughout this disclosure, it is to be understood that anyconnection or communication between any two computers or devices may bephysical or wireless. Once the cartridge is inserted into the desktopslide scanner, a user may instruct the desktop slide scanner to preparea pre-scan of a slide in the cartridge. The pre-scan is at a relativelylow magnification to ensure that the entire tissue sample in the slidefalls within the borders of the scan area. The pre-scan is formed from amosaic of image tiles which are sequentially uploaded to the networkserver as each image tile is scanned. The network server stitchestogether each image tile as it is received until the entire pre-scanimage is formed.

It should be noted that by uploading the image tiles forming thepre-scan to the network server in real-time, it is possible to generatea complete pre-scan image without ever saving the pre-scan image locallyon the acquisition computer. In other words, the acquisition computermay act as an intermediary for acquiring the image data and uploading itto the network server without ever storing, processing or analyzing theimage data. The network server, on the other hand, handles all of thecomputationally intensive operations on the image data. Thisnetwork-based structure enables conservation of computational resourcesby centralizing the most computationally intensive operations on thenetwork server, thereby enabling the acquisition computer and clientcomputer to be less powerful than would be necessary if they wererequired to locally store and process the image data.

As the network server receives each of the image tiles and stitches themtogether to form the pre-scan image, it is analyzing the image tiles inreal time to automatically identify a local area of interest one theslide. By identifying areas of high contrast in the pre-scan image thenetwork server determines the edges of the tissue sample on the slide.The network server may also be programmed to automatically identifyfeatures or anomalies within the tissue sample. Alternatively, a user ofa client computer connected to the network server via a computer networkmay manually identify a local area of interest on the slide. The usermay choose either the automatically or manually identified local area ofinterest for further analysis at higher magnification. Additionally, theclient computer may include software for automatically identifyinganomalies or other areas of interest in the tissue sample.

Once the local area of interest is identified either automatically ormanually, the network server sends instructions to the acquisitioncomputer to commence scanning the local area of interest at a highermagnification. The desktop slide scanner scans the local area ofinterest by sequentially scanning a mosaic of local image tiles. Theacquisition computer uploads each local image tile to the network serveras each local image tile is scanned. The network server stitches eachlocal image tile together as the image tiles are being uploaded. Onceall local image tiles have been uploaded and stitched together, a rawfinal image of the local area of interest is produced.

In some embodiments, because the local image tiles are uploaded to thenetwork server in real-time as they are scanned, a user of a clientcomputer connected to the network server via a computer network is ableto view the local image tiles in real-time. In other words, a user ofthe client computer, for example a pathologist, can view the raw finalimage as it is stitched together piece by piece even though the localimage tiles are not saved on the client computer. This enables the userto more quickly identify anomalies in the tissue sample because it isnot necessary to wait until the entire raw final image is scanned beforethe individual local image tiles can be viewed. However, in otherembodiments the local image tiles are not viewable by a user of theclient computer until all local image tiles are uploaded to the networkserver and mapped with coordinates. In such embodiments, the lag timebetween beginning the upload of image tiles to the network server andviewing/analyzing the final image on the client computer may beapproximately 3-5 minutes depending on image size and bandwidth.

The network server may optionally automatically sharpen each of thelocal image tiles as they are uploaded to form a sharpened final image.The user of the client computer is then able to view both the raw finalimage and the sharpened final image even though neither final image isstored locally on the client computer. By providing images that areautomatically sharpened using known image processing methods,identification of anomalies in tissue samples may be facilitated in somecases.

Additionally, the client computer may be equipped with software foranalyzing and annotating the images stored on the network. For example,the software may enable a user to annotate features of the image formore detailed analysis. The user may move an onscreen pointer todifferent areas of interest in an image of a sample and digitally markthose areas and optionally enter notes detailing why the area should beanalyzed in more depth. In this way the client software allows the userto generate a list of areas of interest that can be accessed by theacquisition computer over the computer network. In some embodiments,these areas of interest may also be physically marked on the slide inthe desktop scanner using a built-in ink ejector that places a smalldrop of ink on each identified area of interest.

When directed to do so by the user of the client computer, theacquisition computer can then direct the desktop slide scanner toautomatically scan each area of interest identified in the list at ahigher magnification so that the user can then analyze those areas ingreater detail. By only scanning the areas of interest at a highermagnification and not scanning the entire slide at higher magnification,computational resources are conserved and the total scan time is greatlyshortened. Of course, it is also possible to scan the entire slide atthe higher magnification if the user of the client computer wishes toanalyze the entire sample in greater detail.

A sample architecture for a network-based pathology system with desktopslide scanner will now be described with reference to FIG. 1. Pathologysystem 10 includes desktop slide scanner 20 connected acquisitioncomputer 22 which is connected to network server 24. It should be notedthat any connected between devices in pathology system 10 may be wiredor wireless, and further that the connections between devices maycomprise local networks. Network server 24 is connected to computernetwork 30 which, in some embodiments, may be the internet. Alsoconnected to computer network 30 is client computer 26 which includesuser interface software that allows a user of client computer 26 toupload and download data from computer network 30. For example, in someembodiments the user interface software may include a web browser withone or more extensions, plug-ins, add-ons, or other embedded softwarethat enhances the ability of client computer 26 to interact with and/orcontrol one or more other devices in pathology system 10.

Network server 24 may include software for analyzing, editing andmodifying data received from acquisition computer 22. For example,network server 24 may include software for analyzing image data toidentify areas of interest in an image and software for sharpening animage, improving image contrast or otherwise modifying an image tofacilitate image analysis by a user of client computer 26. Computernetwork 30 may include storage devices for storing large data filesuploaded to computer network 30 from network server 24. Computer network30 may also include additional computers for processing data uploaded bynetwork server 24 to computer network 30 in order to distribute thecomputational power required for data analysis and processing.

Pathology system 10 also includes slide cartridge 40 which holds one ormore sample slides 42. Each sample slide 42 includes a tissue sample 44which is placed on each slide 42 by a technician, physician orpathologist. Slide cartridge 40 is inserted into desktop slide scanner20 which includes a microscope and digital imaging device for magnifyingand digitizing images of tissue samples 44 on sample slides 42.

A first method of using pathology system 10 will now be described withreference to FIG. 2. Method 100 begins by providing an acquisitioncomputer and a client computer both in communication with a computernetwork (102). A desktop slide scanner in communication with theacquisition computer is also provided (106). A microscope slide with atissue sample is inserted into the desktop slide scanner formagnification of the tissue sample and providing a digital image of thetissue sample (110). The acquisition computer directs the desktop slidescanner to scan the tissue sample on the microscope slide by dividingthe viewable area of the microscope slide into a grid and sequentiallyscanning image tiles which when digitally stitched together form apre-scan taken at a first magnification and first resolution (114). Theimage tiles are sequentially uploaded by the acquisition computer to thenetwork server in real time as the image tiles are obtained by theacquisition computer from the desktop slide scanner (118). The networkserver may analyze image tiles while the pre-scan image is beinggenerated and stitched together to identify areas of interest in thepre-scan image in real time (122).

The acquisition computer then directs the desktop slide scanner to scanidentified areas of interest in the tissue sample at a secondmagnification higher than the first pre-scan magnification (126). Eachlocal area of interest is divided into a grid so that the local scan ofthe area of interest comprises a plurality of local image tiles that aregenerated and uploaded to the network server sequentially (130). Thenetwork server stitches the local image tiles together as they areuploaded (134). A user of the client computer is able to view the localimage tiles in real time as the raw final image of the local area ofinterest is being stitched together and uploaded by the network server(138). A raw final image comprising a mosaic of the assembled localimage tiles is generated and saved on the network server independent ofwhether the raw final image is saved on the acquisition computer (142).

Another method of using pathology system 10 will now be described withreference to FIG. 3. Method 200 begins by providing an acquisitioncomputer and a client computer both in communication with a computernetwork (202). A desktop slide scanner having a microscope stage isplaced in communication with the acquisition computer (206). Amicroscope slide with a tissue sample is inserted onto the stage of thedesktop slide scanner for magnification of the tissue sample andproviding a digital image of the tissue sample (210). A pre-scan of thetissue sample on the microscope slide is taken at a first magnificationand uploaded to the network server by the acquisition computer (214). Auser of the client computer remote views the pre-scan over the computernetwork and analyzes the pre-scan to identify areas of interest in thepre-scan (218). The desktop slide scanner moves the microscope stage tofocus on the identified areas of interest at a second magnificationhigher than the first pre-scan magnification (222). The acquisitioncomputer uploads a live stream of the identified areas of interest(226). The user of the client computer views the live stream in realtime and remotely instructs the digital slide scanner to move themicroscope stage so that different portions of the identified areas ofinterest can be viewed and analyzed in real time (230). The user of theclient computer may also remotely instruct the desktop slide scanner tofocus at different depths of the tissue sample in real time (234).

It should be noted that the phrase “in real time” as used above meansthat instructions from the client computer to the acquisition computeror desktop slide scanner are carried out immediately as they are made bythe client computer and received by the remote device. The phrase “inreal time” also means that image data uploaded to the network server bythe acquisition computer is viewable on the client computer as soon asit is received. For example, the user of the client computer mayinstruct the microscope stage to move to different areas and thecorresponding changing images are immediately viewable on the clientcomputer. In other words, a series of different still images are notdownloaded by the client computer; rather, the client computer receivesa stream of image data that instantly changes as the microscope stage ismoved in response to instructions sent by the user of the clientcomputer.

What has been described above includes examples of one or moreembodiments. It is, of course, not possible to describe everyconceivable combination of components or methodologies for purposes ofdescribing the aforementioned embodiments, but one of ordinary skill inthe art may recognize that many further combinations and permutations ofvarious embodiments are possible. Accordingly, the described embodimentsare intended to embrace all such alterations, modifications andvariations that fall within the spirit and scope of the appended claims.Furthermore, to the extent that the term “includes” is used in eitherthe detailed description or the claims, such term is intended to beinclusive in a manner similar to the term “comprising” as “comprising”is interpreted when employed as a transitional word in a claim.

What is claimed is:
 1. A method for processing, saving and viewing adigital image of a sample comprising: providing an acquisition computerconnected to a network server in communication with a computer network;inserting at least one sample into a digital scanner connected to theacquisition computer; the acquisition computer generating a first imageof the sample at a first magnification and resolution; the acquisitioncomputer uploading the first image to the network server; the networkserver analyzing the first image while the first image is beinggenerated to identify an area of interest in the first image; theacquisition computer generating a local scan of the area of interest ata second magnification higher than the first magnification, the localscan comprising a plurality of local image tiles; the acquisitioncomputer uploading each of the plurality of local image tiles to thenetwork server while the local scan is being generated, each local imagetile being viewable by a client computer in communication with thecomputer network in real time while the plurality of local image tilesare being uploaded; the network server stitching each local image tiletogether as the image tiles are being uploaded and assembling a rawfinal image of the local scan from a mosaic of the plurality of localimage tiles; and saving the raw final image on the network serverindependent of whether the raw final image is saved on the acquisitioncomputer.
 2. The method of claim 1, further comprising: sharpening eachlocal image tile to create a plurality of sharpened local image tileswhile the plurality of local image tiles are being uploaded; the networkserver assembling a sharpened final image from the plurality ofsharpened local image tiles while the plurality of local image tiles arebeing uploaded; and the client computer selecting between the raw finalimage and the sharpened final image for viewing immediately withoutsaving the raw final image and the sharpened final image locally andwithout transferring the raw final image or the sharpened final imageafter the acquisition computer generates the local scan.
 3. The methodof claim 1, wherein the digital scanner comprises a microscope and astage.
 4. The method of claim 1, wherein the identifying the area ofinterest comprises analyzing the first image at a low resolution in realtime to automatically identify the area of interest.
 5. The method ofclaim 1, wherein the sample is a tissue sample.
 6. The method of claim5, wherein the tissue sample is from a patient.
 7. The method of claim1, wherein a user of the client computer views the local tile images inreal time.
 8. The method of claim 7, wherein the user is a pathologist.